Comments for MEDB 5510, Week 05

Topics to be covered

  • What you will learn
    • What is quality improvement?
    • Framework for quality improvement studies
    • Fishbone diagrams
    • Pareto charts
    • Simple quasi-experimental studies
    • Interrupted time series designs
    • Withdrawal designs

Objectives

  1. To contrast the features of a quality improvement study with a research study

  2. To describe the various quasi-experimental approaches

Different names for the same thing

  • Agile
  • Continuous Quality Improvement (CQI)
  • Kaizen
  • Lean
  • Quality Control (QC)
  • Six Sigma
  • Statistical Process Control (SPC)
  • Total Quality Management (TQM)
  • Different from Quality Assurance

Development of quality improvement methods

  • Historical roots
    • Walter Shewhart (1920s, General Electric)
    • W. Edwards Deming (1950s, Japan)
    • Brent James (1990s, Intermountain Health Care)

Process underlying quality improvement

  • Systematic approach
    • Commitment to teams
    • Organization-wide support
    • Passion for measurement

Quality improvement versus research

  • Differences from research
    • Systems approach
    • Little or no attention to generalizability
    • Continuous and cyclical process
    • Major reliance on quasi-experimental studies

Break #1

  • What you have learned
    • What is quality improvement?
  • What’s coming next
    • Framework for quality improvement studies

The SMART approach

  • SMART
    • Specific
    • Measurable
    • Achievable
    • Relevant
    • Time Bounded
  • [Who] will do [what] resulting in [measure] by [when]
    • Minnesota Department of Health

The PDSA cycle

  • Plan
  • Do
  • Study
  • Act

Process, outcome, and balancing measures

  • Outcome measures
    • Direct measure
    • Low signal to noise ratio
  • Process measures
    • Delivering what you promised
    • Understanding the WHY
  • Balancing measures
    • Unintended consequences

Break #2

  • What you have learned
    • Framework for quality improvement studies
  • What’s coming next
    • Fishbone diagrams

Drawing a fishbone diagram, 1 of 3

First step in drawing a fisbone diagram

Drawing a fishbone diagram, 2 of 3

Second step in drawing a fisbone diagram

Drawing a fishbone diagram, 3 of 3

Third step in drawing a fisbone diagram

Fishbone example, 1 of 4

Fishbone diagram

Fishbone example, 2 of 4

Fishbone diagram

Fishbone example, 3 of 4

Fishbone diagram

Fishbone example, 4 of 4

Fishbone diagram

Break #3

  • What you have learned
    • Fishbone diagrams
  • What’s coming next
    • Pareto charts

The Pareto chart

  • Based on the Pareto 80-20 principle.
    • The “frequent few”
  • Proportion of cases associated with a specific cause.
    • Combined with cumulative frequency

Example of a Pareto chart, 1 of 3

Example of a Pareto chart

Example of a Pareto chart, 2 of 3

Example of a Pareto chart

Example of a Pareto chart, 3 of 3

Example of a Pareto chart

Break #4

  • What you have learned
    • Pareto charts
  • What’s coming next
    • Simple quasi-experimental studies

What is a quasi-experimental study?

  • Could but does not use randomization
  • Never sneer at quasi-experimental studies
    • Make a loud mistake
  • Problems with randomization
    • Cost
    • Logistical constraints
    • Contamination
    • Small n
    • Difficult to get buy-in

Notation for research designs

  • O means a measurement is made
  • X means an intervention is given.
  • ~X means no intervention or a control intervention
  • R means randomized assignment
  • NR means non-randomized assignment
  • E means the experimental group
  • C means the control group

Example of a design

\(\ \)

R E: O1 X O2   O3
R C: O1   O2 X O3

Single group post-test only design

\(\ \)

NR E: X O

\(\ \)

  • Simplest design
  • Useful for pilot work

Single group comparison post-treatment to baseline

\(\ \)

NR E: O1 X O2

\(\ \)

  • Allows a comparison.
  • Confounded with temporal trends.

Two group comparison, without a baseline

\(\ \)

NR E: X O
NR C:   O

\(\ \)

  • Non-randomized comparison
  • Confounded with baseline imbalance

Two group comparison with a baseline

\(\ \)

NR E: O1 X O2
NR C: O1   O2

\(\ \)

  • Best design so far.
  • Can check for
    • temporal trends in the control group.
    • baseline imbalances
  • Cannot check for unmeasured covariates
  • Cannot check for treatment interaction

Break #5

  • What you have learned
    • Simple quasi-experimental studies
  • What’s coming next
    • Interrupted time series designs

Interrupted time series design

\(\ \)

NR E: O1 O2 O3 X O4 O5 O6

\(\ \)

  • Best with three or more measures at baseline
  • Check for most temporal trends

Hypothetical patterns in the interrupted time series design

Phased design, 1 of 2

\(\ \)

NR E: O1 O2 X1 O3 O4 O5 X2 O6 O7 O8 X3 O9 O10

\(\ \)

  • Split intervention into three or more pieces
  • Phase in the intervention piece by piece

Phased design, 2 of 2

Stepped wedge design, 1 of 2

\(\ \)

NR E1: O1 O2 X O3 O4 O5   O6 O7 O8   O9 O10
NR E2: O1 O2   O3 O4 O5 X O6 O7 O8   O9 O10
NR E3: O1 O2   O3 O4 O5   O6 O7 O8 X O9 O10

\(\ \)

  • Wait for your turn.
  • Useful for very small sample sizes.

Stepped wedge design, 2 of 2

Break #6

  • What you have learned
    • Interrupted time series designs
  • What’s coming next
    • Withdrawal designs

Withdrawal design (1 of 2)

\(\ \)

NR E: O1 X O2 -X O3

\(\ \)

  • Measure
  • Add the intervention
  • Measure again
  • Withdraw the intervention
  • Measure one more time

Withdrawal design (2 of 2)

Summary

  • What you have learned
    • What is quality improvement?
    • Framework for quality improvement studies
    • Fishbone diagrams
    • Pareto charts
    • Simple quasi-experimental studies
    • Interrupted time series designs
    • Withdrawal designs